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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)S Responsive to communication(s) filed on 30 December 20003 . 
2a)IE This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for fomial matters, prosecution as to the merits is 

closed in accordance with the practice under £xpa/te Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) S Claim(s) 1-19 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)n The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 3S U.S.C. § 119 

12)n Acknowledgment Is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) S Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1 449 or PTO/SB/08) 5) Q Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date . 6) Q Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 



1 . Applicant's arguments with respect to claims 1-17 have been considered but are 
moot in view of the new ground(s) of rejection. 



2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-19 are rejected under 35 U.S.C as being unpatentable over Newman et 

al., US 6,154,600 in view of Liaguno et a!., US 5,729,741. 

In considering claims 1, 6, 10 and 16, 

a) the claimed receiving a video signal is met by the graphical user interface (GUI) (Fig 
14) where the GUI can capture a variety of sources to include, a camcorder 586, VCR 
588, auxiliary audio TV, CD-ROM/DVD and digital camera 590. 

b) the claimed receiving a first audio signal containing annotations is met where the 
graphical user interface can capture additional audio via auxiliary audio (capture 584) 
and also add audio video via storyboard 592, which allows edit, playback and record 
shots, including audio shots (col 14, line 18-22). 

However, Newman does not specifically disclose converting the annotations into 
searchable annotations using a voice-to-text conversion system. 

Newman does disclose a system that can manipulate hypermedia, where 
hypermedia refers to the integration of text, graphics, sound, video and other data, or 



Claim Rejections - 35 USC § 103 
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any combination into a system which can be stored and retrieved (col 5, line 47-52). 
Newman also discloses that audio is synchronized to video by reference to a master 
pixel clock (col 18, line 57-59). 

The conversion of voice (audio, sound) to text, where the converted information 
is searchable is conventional in the art. 

The examiner incorporates Liaguno which discloses a system for storage and 
retrieval of text, pictures, video and audio where the source of information (text, 
pictures, video and audio) are converted to text. Thus Liaguno provides a system 
where a user is able to search (via text) the desired source of information, in order to 
provide the user rapid retrieval of any type of data (col 2, lines 32-42). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Newman which discloses a video home editing system, 
which can edit/cut/insert/reproduce hypermedia including graphics, text and audio 
annotations (col 14, line 17-19), with Liaguno, to also convert the audio to searchable 
text, in order to provide the user the ability to retrieve a desired segment, whether text, 
video, audio etc... of the hypermedia, which would provide the user rapid retrieval of a 
selected segment. 

In considering claims 2, 7, 1 1 and 17, 

However, neither Newman nor Liaguno disclose removing the annotations from 
the second audio signal (signal 120). 
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Newman discloses a media editor where the user is able to 
add/insert/delete/remove hypermedia to an existing hypermedia file by using graphical 
user interface 400 (Fig 9, Fig 14). 

Thus based on the type of hypermedia being edited (i.e. home video), the user 
could create a hypermedia (home video) where the scene/background audio (2"^ audio 
signal) is removed in order to provide additional/auxiliary audio in lieu of 
background/scene audio, thus removing its annotations from the second audio signal. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to modify the combination of Newman and Liaguno which discloses that 
hypermedia can be edited by deleting/removing data (video, text and audio etc), where 
the image data is converted/searchable via text, to form an edited hypermedia medium 
by also deleting the annotations of the second audio signal, in order to provide the user 
the ability to search the desired clips via annotations audio (auxiliary/narrated) or the 
received video. 

In considering claim 3, the claimed utilizing a least-mean square algorittim 
As discussed above in claim 3, neither Newman nor Cruz disclose removing 

annotations from the second audio signal, nor removing by using a least-mean square 

algorithm. 

The applicant states in the specification, that the least-mean square (LMS) 
algorithm is a conventional algorithm, and also (page 12, line 5-12) that annotations 
may be removed by a variety of methods or algorithms. 
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Therefore, it would have been obvious design choice to one of ordinary skill in 
the art at the time of the invention to modify the Newman/Liaguno combination which 
discloses editing and searching hypermedia, by utilizing an algorithm such as a LMS 
algorithm in removing annotations from the second audio signal, in order to provide an 
editing hypermedia which removes undesired audio and allows the user to insert 
additional/auxiliary audio if desired. 

In considering claims 4, 8, 12 and 18, 1) the claimed generating a center text 
title.. . 2) the claimed generating a scrolling text banner. ... 

Newman discloses a system which uses a non-linear home media editor system, 
which allows the user to develop a audio/video information from either multiple devices, 
to include recorder, VCR, TV, CD-ROM/DVD, digital camera, storyboards, internet, 
etc... Newman also discloses a system which lets the user input the type of 
information, i.e. "OUR SUMMER VACATION", which is displayed in the center of the 
window 408 (Fig 12). Newman also discloses the use of scrolling text overlays, by 
moving prerendered graphical objects onto an image from a variety of hypermedia 
sources (col 12, line 17-23). 

In considering claims 5, 9, 13 and 19, 
The claimed generated a video abstract via the first and second audio signals, the video 
signals and the searchable annotations is met where the editing hypermedia created, 
generates a video abstract, where the audio signal(s), video signal(s) and other media 
which is included/added/inserted into the unedited hypermedia is combined to form a 
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desired hypermedia. As shown in Fig 1 1, the edited hypermedia "Vacation" (444), is 
broken down into appropriate storyboards for the user which may be edited/viewed. 

In considering claim 14, 
The claimed wherein the video signal is received from a video recorder is met by 
Newman which discloses receiving video via camcorder 586, VCR 588 and digital 
camera 590 (Fig 14). 

In considering claim 15, 
The claimed wherein the first and second audio signals are received from at least one 
microphone is met where Newman discloses the reception of audio/video via camcorder 
586, VCR 588 and also auxiliary audio. 

Conclusion 

3. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure— please refer to newly cited references on attached form PTO- 
892. 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
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shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

4. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Brian Yenke whose telephone number is (703) 305- 
9871 . The examiner work schedule is Monday-Thursday, 0730-1 830 hrs. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
Supervisor, John W. Miller, can be reached at (703)305-4795. 

Any response to this action should be mailed to: 



Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington, VA, Sixth Floor (Receptionist).Any inquiry of a general nature or 
relating to the status of this application or proceeding should be directed to the 
Technology Center 2600 Customer Service Office whose telephone number is 



Commissioner of Patents and Trademarks 



Washington, D.C. 20231 



or faxed to: 



(703) 872-9314 




(703)305-HELP. 




B.P.Y^ 
March 12, 2004 



